A Lower Bound on the Complexity of Approximating the Entropy of a Markov Source

The Asymptotic Equipartition Property (see, e.g., [3]) implies that, if we choose the characters of a string s of length n independently and according to the same probability distribution P over the alphabet then, for large values of n, the 0th-order empirical entropy H0(s) of s (see, e.g., [4]) will almost certainly be close to the entropy H(P ) of P . Batu, Dasgupta, Kumar and Rubinfeld [1] showed that, if H(P ) = Ω (γ/ǫ), then we can almost certainly approximate H(P ) to within a factor of γ after seeing O (